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ABCmouse Significantly Increases 
Children’s English Language and Digital 


Literacy Skills 


Key Findings 

e Elementary school children in a Spanish- 
speaking country using ABCmouse made 
significantly greater gains in English language 
knowledge in comparison to the control 
group that received business-as-usual English 
language instruction. 


e The more activities students completed on 
ABCmouse, the greater their gains in English 
language knowledge. 


e By using ABCmouse, both students and 
teachers developed greater facility in using 
technology as a tool for learning and teaching. 


e Teachers expressed the desire to continue 
using the program in their classrooms, and 
students described how ABCmouse increased 
their interest and enjoyment in learning English. 


Overview 


Today, more than 1.5 billion people worldwide are English 
learners, and Englishis the most widely spoken and written 
language in the world, with 94 countries where English 
is either the primary or secondary official language. 
English is also the language that dominates the internet, 
and proficiency in English has become synonymous 
with better career opportunities.? Meeting the needs of 
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quality English language education worldwide demands 
extensive customization to local language and culture, 
and there is a need for scalable, affordable resources 
that can effectively serve learners in diverse settings. 


Age of Learning, Inc.'s, ABCmouse Early Learning 
Academy is a comprehensive digital learning resource, 
primarily serving English-speaking children (ages 2-8) 
in the U.S. Age of Learning designed ABCmouse with 
more than 10,000 Learning Activities to help children 
build language and literacy skills, as well as to develop 
knowledge and skills across the subjects of math, 
science, health, social studies, art, and music. 


For this study, Age of Learning partnered with the 
Costa Rican Ministry of Education and Creciendo 
Juntos (Growing Together). Creciendo Juntos is a local 
education nonprofit that serves elementary schools and 
high schools in Guanacaste, a touristic province with 
the highest levels of extreme poverty in the country. A 
primary goal of the study was to test the hypothesis that 
ABCmouse can effectively help children in a non-English- 
speaking context develop English language and digital 
literacy skills. A secondary hypothesis was that through 
ABCmouse, Costa Rican teachers could gain facility in 
using a mobile application to deliver English instruction. 
Age of Learning researchers designed the study and 
worked with educators from the Costa Rican Ministry of 
Education and professionals at the nonprofit, to collect, 
analyze, and interpret the data. 
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Program 


Participating teachers used ABCmouse as a supplement 
to the standard English language learning curriculum in 
Costa Rica. Most teachers had tablets for their students to 
share (although each student had his or her own account 
and Avatar), and they usually supplemented the classroom 
resources with their personal smartphones and hotspots. 
Teachers used ABCmouse in a variety of ways, including 
having children repeat after the sentences in the app, 
having them sing along, reading English books aloud to 
the class, or assigning independent practice activities 
to reinforce whole-group lessons. In collaboration with 
the Costa Rican Ministry of Education, Age of Learning 
curriculum specialists identified Learning Activities within 
ABCmouse that support 1st through 3rd grade Costa Rican 
learning objectives. Curriculum specialists then created 
customized playlists of the Learning Activities best suited 
for English language learning, which teachers assigned 
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Figure 1. Teacher view of a playlist to assign 


to their students. The activities included puzzles, videos, 
songs, games, audiobooks narrated by native English 
speakers, books that students could read independently, 
and comprehension checks following a lesson or 
book. As students navigated through ABCmouse, they 
encountered oral and written English throughout their 
experience, such as instructions for activities, guidance 
while modifying their Avatars, and pronunciations for 
items in their virtual classroom. 


Figure 1 displays a playlist that teachers could assign to 
students as they worked on the Community Helpers unit, 
a topic set by the Ministry of Education. Figure 2 shows 
a book that students could read along with, focused 
on one of the community helpers. Since most children 
were beginner English learners, the curated ABCmouse 
content was comprised of activities originally designed 
for young native English-speaking children in the early 
stages of language development. 
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Figure 2. Sample read-along book for students 





Participants 


A total of 1,043 students and 13 English teachers across 
15 public elementary schools in Guanacaste used 
ABCmouse in school from May to October 2019.7 We 
selected a sample of approximately 20% of the students 
in grades 1, 2, and 3 across the 15 schools (209 students) 
for assessment, of which 197 students participated in the 
pretest and 157 participated in the posttest.* Additionally, 
one elementary school in the same province with 


comparable demographic characteristics as the treatment 
schools provided a sample of one English teacher and 
187 students, of whom 151 participated in the pretest and 
109 participated in the posttest. The final primary analytic 
sample (N = 266) included all students who completed 
both the pretest and posttest (157 treatment and 109 
control students). All teachers agreed to complete a 
pre- and post-program survey, and most participated in 
interviews at the end of the program. A total of 19 parents 
participated in focus groups, as did 24 children. 


3 Six of the thirteen teachers taught at more than one school. The school year in Costa Rica starts in mid-February and ends in early December. 


4 We identified the comparison school after the selection of schools designated to implement ABCmouse; it was not possible to recruit other schools to increase the size 


of the comparison group. 


‘Wi Age of Learning’ 


Design and Procedure 


Treatment group teachers had full access to ABCmouse 
and were asked to use it in their classrooms for at least 
45 minutes per week. Students in the treatment group 
also had access to ABCmouse for home use, but since 
most students did not have internet access at home or 
appropriate devices to use the program outside of school, 
home usage was very limited. Control group teachers 
continued with business-as-usual English language 
instruction, which included the use of resources from the 
internet, books, and some software. 


The primary measure of students’ language skills was 
The Bilingual Verbal Abilities Test (BVAT).2 The BVAT 
is a widely used standardized assessment of bilingual 
individuals’ language abilities, validated across 16 
languages. It contains three subtests: Picture Vocabulary, 
which measures an individual’s ability to produce words 
for familiar and unfamiliar objects; Oral Vocabulary, which 
assesses a person’s capacity to produce synonyms and 
antonyms of target words; and Verbal Analogies, which 
indicates one’s aptitude to comprehend and verbally 
complete logical word relationships. Trained bilingual 


(English and Spanish) assessors administered the pretest 
one-on-one to students at the end of March and the 
posttest in early November 2019. 


Prior to using ABCmouse, teachers participated in an 
eight-hour in-person training, conducted in English and 
Spanish by a senior Age of Learning curriculum expert. 
Teachers also participated in monthly professional 
development webinars. 


On average, all students across treatment and control 
groups had three 45- or 50-minute English lessons per 
week (a total of 2.25-2.5 hours/week). Over the study 
period, treatment group students completed, on average, 
130 ABCmouse activities (SD = 153.4) and spent a total of 
about 11.4 hours (SD = 13.4 hours) using the program or 
26.3 minutes (SD = 30.9 minutes) per week for 26 weeks. 


Results 


We examined the effect of ABCmouse on students’ 
English language skills while controlling for differences in 
students’ pretest scores, grade level, and gender, using a 
method that accounts for the nested structure of the data. 





Finding 1. Students using ABCmouse made significantly greater gains in English language knowledge compared 


to the control group. 


As shown in Figure 3, there were no statistically significant differences between the two groups at pretest, but at 
posttest, Ist and 2nd grade students who used ABCmouse outperformed the control group (p < .05). Third graders 
started out at a significantly higher level at pretest than those in lower grades, and the posttest difference between the 


treatment and control groups was not significant. 
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Figure 3. Average pretest and posttest performance on the English section of the Bilingual Verbal Abilities Test for treatment (1st 
grade = 36, 2nd grade = 51) and control group students by grade (1st grade = 32, 2nd grade = 27). Grade 1 effect size (Cohen’s 
d = .36, p < .05); Grade 2 effect size (Cohen’s d = .26, p = 12). Lighter shades represent pretest scores; darker shades represent 


posttest scores. 


5 Mufoz-Sandoval, A.F., Cummins, J., Alvarado, C. G., & Ruef, M. L. (1998). Bilingual Verbal Abilities tests (BVAT): Comprehensive Manual. Itasca, IL: Riverside 
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Finding 2. The more activities students completed on ABCmouse, the greater their English language gains. 


To understand whether progress within ABCmouse was related to growth in English language skills, we examined the 
relationship between the number of activities completed in the program and students’ performance on the posttest. 
The correlation between the overall posttest English language score and the total number of activities completed on 
ABCmouse is significant, though modest: r= .26 (see Figure 4 below). 
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Figure 4. Correlational relationship between the total number of activities students completed on ABCmouse and their 
performance on the posttest English sections of the Bilingual Verbal Abilities Test (r = .26, p < .001). 


The survey, focus group, and interview data further highlighted the effect of ABCmouse on students’ English language 
learning. Nearly all treatment group teachers (12 of 13) indicated that ABCmouse helped their students expand their 
English vocabulary, and 11 reported that their students’ pronunciation in English improved by using ABCmouse. 
Additionally, teachers indicated that the program helped their students become more interested, motivated, comfortable, 
and confident in learning English (see Figure 5). They further emphasized the value of having students exposed to native 
English speakers through ABCmouse and the variety of activities they can use to reinforce their lesson objectives. 
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Figure 5. Percent of teachers indicating the extent to which they agree that ABCmouse has helped their students develop 
English language skills (1 = “strongly disagree”—5 = “strongly agree”). The percentages represent those who responded “agree” 
or “strongly agree.” 


All 19 parents who participated in focus groups also reported that their children made substantial gains in their English 
language knowledge, especially in vocabulary. Thirteen parents indicated that ABCmouse helped their children become 
more comfortable speaking English, and ten commented on the notable improvement in their children’s pronunciation 
skills since using ABCmouse. 


At the store, the American was looking for some carrots, and he was really lost there, and [my 
daughter] showed him the way and said, ‘Here are the carrots,’ in English.... Then, at church, 
it was a Sunday, there were some American women who greeted her, and [my daughter] said, 
‘Very well, and you?’ and ‘Thanks for coming.’... She really surprised me.” 
—Parent of a 1st grader 


Finding 3. By using ABCmouse, students made notable improvements in using technology as a resource for 
learning. Teachers also reported developing greater facility in using technology in their classrooms. 


ABCmouse had an effect not only on students’ English language skills, but also on their ability to use technology as a 
tool for learning. On surveys completed before and after using ABCmouse, teachers reported on the extent to which 
their students could independently complete a series of tasks involved in using a tablet/mobile device (e.g., finding 
apps, dragging items across the screen, showing others how to use the device). We compared the weighted average 
scores,° indicating students’ level of technology use knowledge before and after using ABCmouse, and the results 
showed a statistically significant difference between the two scores (Pre = 68.5%, SD = 21.7; Post = 75.6%, SD = 11.6; 
t(125) = -2.6, p < .01), indicating that ABCmouse helped students significantly increase their knowledge and comfort in 
using technology (see Figure 6). 


6 The number of students for each teacher served as the weight in the averages. 
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Figure 6. Average level of technology use knowledge before and after using ABCmouse (t(125) = -2.6, p < .01). 


In focus groups, parents spoke with pride about their children’s increased ease in using technology to support their 
learning and claimed that they too were benefiting as their children used ABCmouse. While they generally reported 
having trouble using technology and admitted having limited knowledge of English, 10 of the 19 parents indicated that 
ABCmouse had motivated them to learn English. ABCmouse was a resource for the entire family, and parents enjoyed 
learning and exploring the activities in the program with their children, while gaining greater facility in using mobile 
devices. 


“| have noticed a big change in [my son]. Apart from improving his use of tablets and cell 
phones and technology, he knows how to use them better than I do.... And sometimes they 
give him homework in English... he writes down what the teacher gives him and looks up the 


words on Google translator.” 
—Mother of a 3rd grader 


Even teachers reported that they learned a great deal about using technology through ABCmouse. Of the seven 
interviewed teachers, none had ever used tablets in their teaching; for some, the only technology they had used before 


was a projector. 


“At first, | was afraid because I didn’t know how to use a tablet or computer.... So ABCmouse 
for me is a tool to learn to use technology.... | learned how to use the app, how to use 
the projector, and the computer.... Sometimes | call my colleague to figure out how to do 
something in ABCmouse, and little by little, we are learning. ”? 
—English teacher 
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Finding 4. Students enjoyed using ABCmouse to learn English, and teachers wished to continue using the 
program in their classrooms. Students also attributed their greater interest in learning English to ABCmouse. 


Data collected through surveys, interviews, and focus groups indicated that both teachers and students had a positive 
experience using ABCmouse: 


e Twenty-three of the twenty-four focus group students reported that ABCmouse led them to enjoy 
learning English. 


e Fourteen of the twenty-four focus group students shared that using ABCmouse has inspired them to 
become English teachers when they grow up. 


Students also described how using ABCmouse affected the way they use technology and how they think about their 
future. ABCmouse led them to become more interested in using the internet and other media (e.g., films, television 
shows) to access content in English; they also reported career aspirations, as well as a greater interest in communicating 
with foreigners and traveling abroad. 


Figure 7 shows teachers’ survey responses on items describing their experience with using ABCmouse in their 


classrooms. 
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Figure 7. Percent of teachers reporting “agree” or “strongly agree” with statements about their experience with using ABCmouse 
(1= “strongly disagree”—5 = “strongly agree”). 


When asked if they would recommend ABCmouse to others, all seven interviewed teachers reported that they would 
recommend it to their colleagues as well as to parents. 


*“£ABCmouse is the best tool | ever had teaching English.... And ABCmouse is a great 
application for the family... not only for the kid, because it can involve all the members of 
the family to learn English, to learn mathematics, to learn social studies.... It’s a familial 
application for everybody to learn. ”? 

—English teacher 
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Conclusion 


This is the first study to examine the effectiveness of 
ABCmouse in helping non-native English-speaking 
students develop their English language knowledge. 
Students used a customized set of playlists containing 
content aligned to the English language curriculum set 
by the Costa Rican Ministry of Education. Results showed 
that in comparison to business-as-usual instruction, 
ABCmouse is effective in helping build the English 
language skills of early elementary students who are 
learning English as a foreign language, especially their 
vocabulary and pronunciation skills. Using ABCmouse 
also helped students and teachers to develop greater 
comfort in using technology as a tool to support learning 
and teaching. The results of this study suggest that a 
digital learning program designed for native English- 
speaking children, with modest adjustments and 
collaboration with local partners, educators, and parents, 
can meaningfully improve English language skills in non- 
English-speaking communities. 


